Collaborative remote interactive platform

ABSTRACT

Systems and methods for providing a remote interactive experience are provided. The systems and methods may include computer executable code stored in a non-transitory memory and run on a processor. The remote interactive experience may be provided via a digital platform. The platform may provide management of a shared browser. The shared browser may be shared by a plurality of participant devices. The platform may store the shared browser in a cloud-based location. The shared browser may thus be cloud-based. The platform may enable the plurality of participant devices to access the cloud-based shared browser. The platform may receive, as input, actions performed to each of the plurality of participant devices. The actions may include at least one of an action list that includes: a cursor movement, a window-sizing, a window-positioning, and a file navigation performed to the shared browser. The platform may propagate the actions to the shared browser.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a non-provisional of Provisional Patent ApplicationNo. 63/043,081, filed Jun. 23, 2020, and of Provisional PatentApplication No. 63/128,321, filed Dec. 21, 2020, which are herebyincorporated by reference herein in their entireties.

FIELD OF TECHNOLOGY

Aspects of the disclosure relate to digital platforms. Specifically,aspects of the disclosure relate to digital platforms for remoteinteraction.

BACKGROUND OF THE DISCLOSURE

Over the past twenty years, the way in which people shop for everydayproducts has radically changed. E-commerce is now a $4 trillion industryinvolving nearly every category of products that once could only bepurchased by visiting a physical store. E-commerce allows commerce to beconducted anywhere you have a mobile device, provides price and qualitytransparency, time savings, ability to service the long-tail of demand,and much more.

There is now a large-scale transformation in how people and devicesinteract with each other. As an illustrative example, there is atransition in how service providers interact with their clients in the$8 trillion-plus service sector. The transition is driven by factorssimilar to those that drove the adoption of e-commerce, including:client expectations of being able to do anything from anywhere; moderntechnology platforms that allow people to discover service providersfrom around the world; and factors limiting physical and geographicmobility.

Recently, there has been a significant growth in adoption of videostreaming platforms such as YouTube, Coursera, and others. Videostreaming platforms may be effective for asynchronous communication ofcontent and entertainment, but lack real-time interaction capabilitiesthat many remote session participants require.

Additionally, there has been a rapid adoption of video conferencingtools to attempt to deliver services like group fitness, music lessons,physical therapy, and more. Synchronous video conferencing tools likeZoom, Skype, WebEx, and Google Meet may suffice for conversations andpresenting documents like spreadsheets and presentations. These toolswere designed to replace in-person business meetings and thus lack theinteractivity, personalization, and pre- and post-session experiencethat may be needed by service providers and their clients, as well as byparticipants seeking remote interactive experiences.

For at least these reasons, it would be desirable to provide systems andmethods to enable participants to engage in collaborative, interactivesessions even when the participants are physically remote.

SUMMARY OF THE DISCLOSURE

Systems and methods for providing a remote interactive experience areprovided. Systems and methods may include computer executable codestored in a non-transitory memory and run on a processor. The remoteinteractive experience may be provided via a digital platform.

The platform may receive instructions from a host device to configure avirtual room. The virtual room may be configured to be accessible to oneor more participant devices.

The platform may receive, from a first participant device, a request toaccess the virtual room. In response to the request, the platform mayrender a canvas on a screen of the first participant device. The canvasmay include a customizable graphical representation.

The canvas may be configured at least in part based on the instructionsof the host device. The instructions may include a selection of one ormore interaction apps, from a set of interaction apps, for inclusion inthe canvas.

The one or more participant devices may be associated with a level ofaccess. The level of access may be based at least in part on theinstructions. The level of access may include a set of rules. The set ofrules may define, for each of the one or more participant devices, whichinteraction apps are included in the canvas. The set of rules maydefine, for each of the one or more participant devices, whichinteraction apps the participant device is permissioned to activate. Theset of rules may define, for each of the one or more participantdevices, a level of permission of the participant device to modify aninteraction app.

The interaction apps may include a coordinated window. The coordinatedwindow may be activated by the host device or the first participantdevice. The coordinated window may be initiated with a layout and acontent. The layout may include a size, a shape, and a screen position.The content may include words and/or images that may be displayed withinthe coordinated window. The coordinated window may be rendered on thecanvas of the host device and on the canvas of the first participantdevice with the same layout and content.

When the host device or the first participant device executes an actionto modify the layout or the content of the coordinated window, themodification may be applied to the coordinated windows on the canvasesof both devices.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects and advantages of the disclosure will be apparent uponconsideration of the following detailed description, taken inconjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows an illustrative system in accordance with principles of thedisclosure;

FIG. 2 shows an illustrative apparatus in accordance with principles ofthe disclosure;

FIG. 3 shows an illustrative screenshot in accordance with principles ofthe disclosure;

FIG. 4 shows another illustrative screenshot in accordance withprinciples of the disclosure;

FIG. 5 shows yet another illustrative screenshot in accordance withprinciples of the disclosure;

FIG. 6 shows still another illustrative screenshot in accordance withprinciples of the disclosure;

FIG. 7 shows an illustrative screenshot in accordance with principles ofthe disclosure;

FIG. 8 shows another illustrative screenshot in accordance withprinciples of the disclosure;

FIG. 9 shows an illustrative system architecture in accordance withprinciples of the disclosure;

FIG. 10 shows an illustrative flowchart in accordance with principles ofthe disclosure;

FIG. 11 shows an illustrative system diagram in accordance withprinciples of the disclosure;

FIG. 12 shows an illustrative flowchart in accordance with principles ofthe disclosure;

FIG. 13 shows an illustrative system diagram in accordance withprinciples of the disclosure;

FIG. 14 shows an illustrative system diagram in accordance withprinciples of the disclosure; and

FIG. 15 shows an illustrative system diagram in accordance withprinciples of the disclosure.

DETAILED DESCRIPTION OF THE DISCLOSURE

Systems and methods are provided that enable participants to interactwith each other even when the participants are physically remote. Thesystems and methods may effectively deliver services and/or featuresthat were traditionally only available in person. Service providers andclients may be used herein as one illustrative example of any suitablegroup of participants in any suitable session, meeting, or conference.One or more of the participants may be a host. A service provider may,for example, be a host. Anything described herein in reference toservice providers and/or clients may, in certain embodiments, be appliedto any suitable participant in a remote digital session.

Prior remote teleconferencing systems have a number of deficiencies,especially when these systems involve interaction between participants.Exemplary deficiencies of prior systems include:

-   -   Service providers and clients struggle through overlapping and        contentious video and audio streams.    -   Clients see poor quality multimedia streams played by the        Service Provider. This may include music or video that the        service provider is playing from a 3rd party server that is        first being routed to the service provider's machine, and then        routed over the video connection to the clients.    -   Clients have no control over what they see, when they see it,        what order they view the livestream, and so on.    -   Service providers cannot share interactive tools for the use of        the clients.    -   Service providers cannot deliver a personalized experience to        the customer. It is one size fits all (or one screen-share fits        all).    -   Participants cannot collaborate over a coordinated file where        actions (including content, navigation, and/or layout        modifications) by one participant are synchronized and updated        for all the participants.    -   In group sessions, service providers cannot easily momentarily        focus on one client within a group of clients without cumbersome        options like breakout rooms that separate the service provider        from the other participants.    -   Service providers cannot typically develop digital interaction        methods tailored to the needs of their specific service. For        example, a yoga instructor interacting with their clients has a        very different type of interaction than a music teacher or a        language tutor or a physical therapist.    -   Service providers find it overwhelming to string together        existing technologies and tools in order to provide their        services.    -   Service providers cannot easily package segments of the session        for the consumers to review or practice after the session.    -   Hosts cannot custom configure multiple virtual “rooms” that        persist in their custom configurations, and can be “visited” and        used by the host and invited participants at will.

Aspects of the disclosure provide systems designed to address the issuesidentified above. Exemplary features of the disclosed systems aredescribed in the ensuing paragraphs.

In conventional prior art systems, video is dominant, and other data andmedia forms are subordinate. Disclosed systems provide a canvas on whichone may tailor his or her view of the underlying global data model. Eachuser may render his or her own view of the underlying data.

Live interaction authoring by the Provider: Disclosed systems mayinclude features for hosting a live interaction stream with interactionapps. The interaction apps can be activated and appropriately displayedby the Provider to improve the interaction experience with the clients.For example, one interaction app could provide alternate camera views ofthe Service Provider. In another example, an interaction app for a musicteacher could provide a time synchronized view of sheet music thatclients should follow.

In yet another example, an interaction app for session participants mayinclude a coordinated window. The coordinated window may be displayed onthe screens of some or all of the participant devices that areparticipating in a session. Modifications made to the window by the host(and/or any other participant that is able to modify the window) may besynchronized and generalized to the windows shown on all of theparticipant devices. Modifications may include modifications to thecontent, navigation, size, shape, and/or layout of the window or thecontents thereof.

The coordinated window may be activated by the host. In certainembodiments, some or all of the participants may also be able toactivate a coordinated window. The host may be able to set which of theparticipants are able to activate and/or modify a coordinated window.

A coordinated window may contain and display any other app. Acoordinated window may contain and display a file or document. Forexample, the coordinated window may be activated to display a pdf, jpg,or other suitable file. The window may be initialized on a certain pageof the file (e.g., page 1). The window may be initialized to be acertain size and shape and at a certain location on the screen. Thesize, shape, screen location, and page being displayed may be uniformacross all the participant devices in the session. A participant (e.g.,the host or another permissioned participant) may navigate to page 2.This action may cause the windows on all the devices in the session tonavigate to page 2 of the file. Similarly, a participant may resizeand/or relocate the window on his or her screen. In some embodiments,these actions may also be mirrored by the windows of all the otherparticipant devices. In some embodiments, a participant may be able tozoom in, or crop, the contents of the window. A participant may be ableto execute various other actions to and within the window. Some or allof these actions may be mirrored by the windows of some or all of theother participant devices. A coordinating window may thereby augmentcollaborative properties to a file or document that typically does nothave such properties.

In another illustrative embodiment, a coordinating window may beactivated to display a shared web browser. In another embodiment, acoordinating window may be activated to display a shared document. Theshared document may be cloud based. Content within the shared documentmay be modifiable. Modifications made to the shared document by oneparticipant may be updated and mirrored in the windows of the otherdevices in the session. For example, the shared document may include aword processor. The participants in the session may be able to navigate,view, and otherwise collaborate on text shown in the word processor. Oneparticipant may delete, modify, and/or type new text in the shareddocument, and the shared document may be synchronized and updatedaccordingly across all the devices.

In some embodiments, the host of a session may be able to customize thecoordination settings. For example, the host may be able to permissionsome or all of the participants. The host may also be able to customizewhich actions are generalized and which are not. For example, the hostmay enable content modifications to be generalized, but not navigation,size, or position. The host may enable navigation to be generalized, butnot size or position. The host may enable size to be generalized, butnot position. The host may, in certain embodiments, be able to set anyother suitable custom setting.

The platform may thus provide a tool that is especially useful for acollaborative work, e.g., a group collaborating on a project. Theplatform may include a shared browser. The shared browser may becloud-based. In some embodiments, the platform may, in operation, allowthe multiple participants access to the same shared browser. Theparticipants may be associated with different permission levels toexecute actions to the shared browser, as described herein.

The collaborative platform as described, may, in certain embodiments,operate differently than a synchronized document. A synchronizeddocument may allow multiple users to access and modify a shared file.The modifications may all be applied to the shared file, which may beupdated based on the modifications. Each user may then access theupdated file separately on separate browsers and on separate devices.Such a synchronized file may allow for remote sharing of updates, butmany other collaborative features are lacking.

The disclosed collaborative platform, however, may, in some embodiments,provide the users simultaneous, collaborative access to a single,remote, shared browser. The collaborative platform may provide anexperience that is much more collaborative than a synchronized document.For example, all actions performed by all participants may be propagatedto the shared browser, and by extension to all the participant deviceswhich are accessing the shared browser. This may include inputs,gestures, cursor movements, and modifications. This may also includewindow sizing, positioning, navigating, and any other suitable actionperformed to a shared browser.

The collaborative platform may also provide features that solvechallenges that may be encountered when multiple users access a sharedbrowser. For example, one challenge encountered may involve collisions,and how to coordinate multiple inputs performed on separate participantdevices.

One provided feature may include animating certain actions on the sharedbrowser to notify other users that the actions are being performed on adevice. For example, a clicking action may draw some waves or linesaround the cursor (e.g., like waves in a water puddle. A drag action mayshow some trail lines, as if a small boat is being dragged on water. Theanimations may be subtle and inconspicuous, but may serve to notifyother users to the action (e.g., dragging, clicking, right clicking,shaking, typing) being performed by one user.

Another provided feature may address a challenge that when multiplepeople are using the same app, conflicts of typing, mouse operations,finger gestures or other conflicting input actions may occur. A buffermay be provided that will allow a user who first initiates an action, tocomplete his/her operation (e.g., they may be allowed to finish typingthe sentence). For example, if user 1 starts to type text, the servermay allow user 1 to retain exclusive focus on the browser for a presetamount of time (e.g., 1, 2, 3 seconds, or other suitable amount oftime). Exclusive focus may prevent actions of other users from beingapplied to the browser. In some embodiments, the platform may save theother actions of other users in a buffer, and apply them, in order theywere performed, when the exclusive window of user 1 terminates.

Moreover, the platform may track the buffered actions, and apply them asappropriate to the current state of the browser. For example, if user 2clicked on a certain location on the browser (e.g., a word or button)while user 1 has exclusive focus, and during the exclusive focus user 1navigates the content in the browser such that the word or button is nolonger in the same location as when user 2 performed the click, theplatform may apply the click to the word or button even though it is ina different location than when user 2 performed the click.

In certain embodiments, or for certain inputs, the platform may beconfigured to combine inputs received from multiple users into acombined input for the shared browser. For example, when the inputincludes audio input received from microphones on the user devices, theplatform may be configured to combine the audio inputs and apply thecombined audio input to the browser. The combined audio input may belater parsed into its constituent parts. This may be useful, forexample, when the platform employs an AI-based language processing toolto receive and act upon audio instructions.

Another challenge that may be addressed by the collaborative platformrelates to security. For example, a security concern may arise when oneuser signs into a site that contains, or has access to, sensitiveinformation. This may present an opportunity for other users to access,and possibly compromise, that sensitive information. A solution may beprovided in the form an app level feature that tracks the owner of thesensitive information. For example, if user 1 signs into a site withsensitive information, the platform may detect when user 1 leaves theroom. In response, the system may automatically log out of the sensitivesite. The system may close the application. The system may send theowner of the sensitive information an alert that his/her information isvulnerable to being accessed by the other users.

In some embodiments, even when the user is in the room, securityprecautions may be taken to protect sensitive data. For example, a usermay have the option, when he/she logs into a sensitive site, to preventother users from performing actions on the site. An option may also beprovided to allow users to perform options to the page being accessed,but may prevent other users from navigating away from that page andaccessing other, possibly sensitive, pages.

Another contemplated feature may include providing an option for a userto retain a private portion of the canvas. A user may thereby have aprivate portion of his/her canvas that is not a part of the sharedbrowser, alongside a public portion of the canvas that is based on theshared browser and is shared across all devices.

Another feature of the platform may enable a user to transfercredentials and/or identities from a local system to the shared browser.For example, if a user is signed into and authenticated by a securesystem on his/her local device, the local authentication may betransferred to the shared browser without the need to reauthenticate.

In some embodiments of the platform, a session may be initiated whenparticipants join a room. Apps may be added, activated, or used once thesession is under way. In other embodiments, the platform may beconfigured to be initiated from within a document or website. Forexample, a button may be provided which, when a file or webpage is open,launches the file or webpage in a canvas, and invites participants tojoin and collaborate on the webpage or document. For example, a travelplanning website may be set up to have an integrated button which a usercan select. When the user selects the button, he/she may be able toinvite other users to collaborate on planning a trip. Other websiteswhich may benefit from such a feature may, for example, includee-commerce sites, software as a service (SAAS) sites, and sites thatprovide support and training.

In other embodiments, the button may be provided as a browser add-on soa user can create an interactive session from any webpage in thebrowser. In yet another embodiment, the button may be an icon on acomputing device which may be able to create an interactive session fromany window opened on the device.

An exemplary process flow may include: a button (which may, in someembodiments, say something like “together” or some similar message whichmay convey the function of creating a collaborative session) ispresented on a web site, SAAS app (source URL), browser extension, ordesktop icon. A user may select the button. Selecting the button maytrigger a session room to initiate. The configuration of the room may bepreset by the website or the user. The room may initiate with a sharedbrowser with the URL of the website from which the user selected thebutton. The user may be able to invite others to the room. In somecases, the user may invite support from the company's customer supportpersonnel. In those cases, the invite operation may place the user in aqueue until a representative is available to join.

In some embodiments of the platform, the host participant maypreconfigure one or more virtual “rooms.” Each room may be configuredwith custom default settings. The settings may relate to the look (e.g.,background, colors, etc.) and/or functionality (e.g., audio and videosettings, apps that are included in or can be added to the canvas,permissions for what some or all the participants are capable ofaccessing, activating, or modifying, etc.). The room configurations may,in certain embodiments, be persistent. For example, a host may configurea number of rooms with custom settings, and the settings may be saved,and may remain the default settings for those rooms for future sessions.

The host may invite the participants to a room. Each participant may beassociated with a participant device. An invitation may include auniform resource locator (URL) link, a room ID and/or password, or anyother suitable invitation. A participant may access the room by clickingon the link and/or following the URL. A participant may access the roomby navigating to a website. A participant may access the room via anadd-on to a browser that may provide a shortcut to access the room. Aparticipant may access the room via a dedicated app on a computingdevice or smart phone.

Upon accessing the room, each participant device may display a canvaswith one or more apps according to the configuration of the host. Forexample, the canvas displayed on the screens of all the participantdevices may include video icons displaying video feeds from cameras oneach of the participant devices (along with audio from microphones onthe participant devices). The canvases might also display buttons foractivating other apps. The buttons may be displayed on the host canvas.The host may configure the session so that the buttons are alsodisplayed on the canvases on some or all of the participant devices. Thebuttons may include a button for activating a coordinated window, abutton for activating a shared browser, a button for activating a shareddocument, and/or any other suitable button for activating any other appor feature that may be available for the canvas. For example, thebuttons may include a button for activating a share screen. Selecting ashare screen button may open a window that provides options forselecting one or more windows running on the selector's device forsharing.

In certain embodiments, interaction apps may be developed and initiatedby the Service Provider (SP), or another user configuring a canvas. Appsmay also be developed by third party developers taking advantage ofsuitable APIs. Apps may, in some embodiments, be included in adownloadable version of the platform. Apps may, in some embodiments, beaccessed in an app store. In certain embodiments, some Apps may beincluded by default as part of a basic platform package, and the basicpackage may be augmented by creating and/or downloading additional Apps.

Exemplary interaction apps may include apps for music, languagelearning, group exercise, group lessons around games and sports, andother suitable areas.

Interaction Apps may also include website embedding, voting, sharefolders, metronome, trivia, notes, marking/bookmarking video,mirror/recording, shared games (e.g., Chess), multiple cameras on eitherside, practice programs and reminders.

Live interaction composition on the client side: A client (otherwisereferred to herein as a participant) in a live interaction streamingsession may, in certain embodiments, choose his or her own interactionexperience. A client may choose which of the interaction app streamsthey want to engage with and/or how those interaction app streams arearranged on his or her viewing device.

In certain embodiments, the client's viewing device could compose anoptimized interaction experience based on the local computationalresources, network connectivity, and user preferences. For example: Aninteraction app may instruct the clients to view a YouTube video, anddirect the client device to stream the video from a server close to andwell connected to the client, eliminating the need for the ServiceProvider to stream the video from his or her local computing devicethereby introducing delays and bandwidth related video/audio qualityproblems.

Client attention moments: During a group interaction session, certainembodiments of the platform may seamlessly enable a Provider tomomentarily pay attention to, focus on, and interact with one client,for instance in a secondary interaction stream, while allowing the restof the clients in the group session to continue viewing a primaryinteraction stream.

Practice sessions: In some embodiments, a client may be able to recordhis or her interaction with his or her canvas (e.g., a practicesession), go back in time to review and/or correct a portion (e.g., 10,15, or 30 seconds, or any other suitable amount of time) of theinteraction, annotate and tag the interaction stream, capture multipleattempts, and/or share portions of the interaction stream with his orher Provider. Annotations and tagging could be achieved via voiceinterface, touch sensitive screens, or text annotations. In anotherexample, a Provider can join the client session stream to interactdirectly with the client, including recording and annotating portions ofthe client's session stream directly on the client's behalf In yetanother example, the Provider may be able to review and annotate aportion of a client stream (e.g., a practice session), and, in realtime, deliver the annotated portion to the client.

Interactive teaching agents: A SP and/or a client in a practice sessionstream may, in certain embodiments, initiate an instance of anautonomous practice training agent that monitors the practice sessionstream to capture the quality of the client's performance of theassigned practice task. The agent may provide real-time feedback to theclient and/or the SP. In another example, the Service Provider couldprovide tests or exercises for the client.

Interactive practice session dashboard: In certain embodiments, theplatform may include a dashboard for the Provider to view performancemetrics. The dashboard may display performance metrics of one or moreparticipants. The dashboard may be customizable. The dashboard maydisplay the metrics in real-time and/or as part of a post-sessionreport. The performance metrics may include information generated byautonomous training agents. The performance metrics may includeinformation about participant response to tests or other exercises.

Asynchronous interaction stream replay: In some embodiments of theplatform, each interaction app may generate an Interaction Data, Eventand Action stream (IDEA stream). Each of these IDEA streams may becaptured, time-synchronized and stored together. The IDEA streams mayinclude session data segmented and compartmentalized into separatelyidentified portions. For example, a camera video feed (which maycorrespond to one interaction app selected to be part of a canvas) maybe one portion. A slideshow playing on another app may be a separateportion. A quiz given via another app may be yet another portion.Furthermore, metadata may be generated and saved along with the datastreams. The metadata may identify and categorize the data contained ineach data stream. Separating and identifying the data in this way mayenable organization and access to data that may not be feasible withconventional systems. Clients and Providers may be able to replay theinteractive sessions, and may be able to compose a personalizedinteraction experience. The personalized experience may include choosingone or more of the IDEA streams to view and/or arranging the streams ina way that best suits that viewer and viewing device.

Indexing of live interaction stream by navigation actions of Providerand Clients: During a session, a Provider and/or a client may navigatebetween sections of composed interaction apps, and may also navigatewithin the content of individual composed interaction apps. Thesenavigation actions may form part of the captured IDEA streams.Navigation actions may be selectively extracted into an index of thecaptured IDEA streams. The index serves as a time-linked “table ofcontents” and can be personalized for individual users. The indexedactions may be represented hierarchically based on whether they'rerecorded from within-interaction-app navigation, between-composed-appsnavigation, or other hierarchy inherent in the content captured in theIDEA streams.

Exemplary Implementations

Exemplary implementations of systems with above-mentioned features aredescribed below. The exemplary implementations are provided for purposesof illustration rather than limitation.

Disclosed systems may include at least three entities: A “User Device,”which may include a processor, network interface, video and audio inputdevices, display and audio output devices, additional input/outputdevices, and/or other suitable elements. A “Coordinating Server,”(otherwise referred to herein as a central, or cloud-based, server)which may include a processor, storage device, network interface, and/orother suitable elements. A “Content Delivery Server,” which may includea processor, storage device, network interface, and/or other suitableelements.

One User Device, associated with a user distinguished as the ServiceProvider (alternatively referred to as a Host), may initiate a Sessionby contacting a Coordinating Server through its network interfaces. TheCoordinating Server may create an initial, durable Session Record on astorage device. The creation of the Session by the SP (Host) may becaptured by a Session Event that may be added to the Session Record.

One or more additional User Devices, associated with users denotedParticipants, may join the Session by contacting a Coordinating Serverthrough their network interfaces. The Participants may, in certainembodiments, be authenticated before joining the session. Authenticationmay be achieved via a passcode, biometric identification, login from atrusted device or account (e.g., from a set of devices or accountsauthorized by the host), and/or any other suitable form ofauthentication. The connection of a Participant to the Session may becaptured by a Session Event that may be added to the Session Record.

The display output devices of connected User Devices may be coordinatedat least in part via Canvases. At the beginning of a Session, the SP(Host) and Coordinating Server may configure an initial Canvas. A Canvasmay include a graphical representation with customizable content and/orlayout. Canvases may be configured to include content, such as selectedApps and/or other features. Canvases may have associated properties suchas title, aspect ratio, background, and others. The Host may configureone Canvas for display on all participating devices throughout thesession. In some embodiments, the host (or, optionally, one of theclients) may configure multiple canvases for display on differentparticipant devices and/or sequentially on the same devices. In someembodiments, all canvases are pre-configured prior to a session. Inother embodiments, one or more of the canvases may be configured and/ormodified mid-session.

Creating and/or loading a subsequent canvas may be a result of anexplicit action. A user configuring canvases may be presented withmultiple action options. Exemplary action options include: “Next”: whenusing a sequence of pre-composed canvases, switch to the next one in thesequence; “New”: make a freshly-created, default canvas the next one;“Duplicate”: make a new canvas and copy the layout of the current canvasinto the new one; and “Insert Template”: make a new canvas from alibrary of pre-composed templates.

A Session Event representing the appearance of the initial Canvas andits content may be added to the Session Record. During the Session, aUser Device—usually associated with the SP (Host)—may contact theCoordinating Server to replace the current Canvas with a new or modifiedCanvas. The appearance of the subsequent Canvas, its properties, and itscontent may be captured by a Session Event that's added to the SessionRecord. The Coordinating Server may notify connected User Devices of theappearance of the new Canvas, and the User Devices' display outputdevices may update to show the content of the new Canvas. The content ofthe new Canvas may have been composed before initiation of the Session;for example, imported from the Session Record of a previous Session.Pre-composed Canvases may be referred to herein as Templates.

Some exemplary interaction apps and other exemplary preferred featuresof the disclosed systems are described below in further detail.

Interaction Apps

Canvas configuration may include content (e.g., one or more Apps)arranged into a layout: sizes, positions, z-ordering, and otherproperties that determine display of Apps relative to each other and theCanvas itself. Canvases can be composed and rearranged during Sessionsby a User Device—usually the SP (Host)—sending a request to theCoordinating Server to update properties of the Canvas and its layout,including adding and removing Apps. The change to Canvas and/or Applayout properties may be captured by a Session Event that may be addedto the Session Record.

Apps may include a User Device-side component and (optionally) aCoordinating-Server-side component. Code modules within the UserDevice-side component incorporate a User Devices' view of the globaldata model for the Session and output visual and/or audio content to theUser Devices' output devices. Coordinating Servers may send streams ofevents to User Devices over their network interfaces. Events in thestreams usually result in User Device-side Apps updating visual andaudio output, often because an event notifies an update to the globaldata model.

Different User Devices may have different authenticated identities androles—including, for example, SP (Host) and one or moreParticipants—within Sessions. User Device Apps' views of the global datamodel may be personalized to each User Devices' identity and role.Because of this personalization, Apps for different User Devices may ingeneral receive different streams of events from Coordinating Serverswithin the same Session.

User Device Apps may also send outgoing streams of events toCoordinating Servers. In Coordinating Servers, the incoming streams ofevents from User Device Apps are received and processed by the secondcomponent of Apps, the Coordinating-Server-Side components. Uponreceiving an incoming event from a User Device App component,Server-Side Apps may update the global data model and User Devices'views of the model, and initiate the sending of events to User DeviceApps in return. Received events and effected data-model changes may becaptured as Session Events which may be added to the Session Record.

Live Stream App

One example of an Interaction App is Live Stream, which streams from onevideo source and/or one audio source of a User Device. A video sourcemight be an input video device like a camera, or a video stream of aUser Devices' “screen”, i.e. screenshare, or others. An audio sourcemight be an input audio device like a microphone, or others. Video andaudio sources might include multiple tracks, for example stereo audio orstereo (3D) camera. The unit of data sent from User Devices' sourcestreams may be called a Frame. A Frame may include image and/or audiodata.

The User Device-side component of Live Stream App may create the UserDevice video source stream in order to receive incoming Frames. Uponreceiving a new Frame from the source stream, the User Device-sidecomponent of the Live Stream app may:

-   -   Optimistically update its local view of the global data model to        cache the newly-received Frame and output it to display and/or        audio-playback device(s); and/or    -   Send an event to Coordinating Server(s) that includes the new        Frame and other associated metadata.

Server-side components of the Live Stream App—and possibly otherServer-side App components—may receive the incoming event. Server-sidecomponents of the Live Stream App may:

-   -   Use the incoming Frame data to update originating User Devices'        information in the global data model;    -   Update other User Devices' views of the global data model;    -   Send events to User Devices whose views of the global data model        changed in a previous step; and/or    -   Capture the incoming event from the User Device as a Session        Event and add it to the Session Record.

After the Coordinating Server sends events to User Devices whose viewsof the global data model changed, the User Device-side Live Stream Appcomponent for each User Device may receive the event and in returnpossibly outputs Frame data to local display and/or audio devices.

YouTube Player App

Another example of an Interaction App is YouTube Player, which maystream video and audio directly to User Devices from Content DeliveryServers. A User Device—usually the SP (Host)—may add an instance ofYouTube Player to a Canvas and may parameterize it by the YouTube videoURL and/or other properties. This User Device may be denoted theControlling User Device. The Controlling User Device may send to aCoordinating Server an event that bundles a representation of the newYouTube Player instance and any changes to Canvas properties thatresulted from including YouTube Player in the layout. CoordinatingServer may capture the incoming event as a Session Event and may add itto the Session Record. The event may then be forwarded to the other UserDevices—usually Participants—who may update their copies of the datamodel and render the new YouTube Player instance to an output displaydevice, along with other changes to the Canvas.

The Controlling User Device may begin playback of the video by an actioninitiated through an input device, for example a “mouse click”. Theaction may be sent to Coordinating Server where it may be: (i) capturedas a Session Event and added to Session Record; (ii) used to updateglobal data model; and/or (iii) forwarded to other User Devices—usuallyParticipants. As User Devices receive updates to the global data model,playback of the YouTube video may be initiated. Each User Device mayindependently dial Content Delivery Servers to negotiate optimal videoand audio streams given the User Device's processor(s), networkinterface(s), quality-of-service (QoS) preferences for e.g. lowerlatency vs higher quality, output display and audio devices, geographiclocation, and/or any other relevant environmental information.

During playback, the Controlling User Device may alter playback state byactions such as pausing at a timepoint, muting audio, seeking to atimepoint, and others. These actions and relevant properties may be sentto the Coordinating Server, captured as Session Events in SessionRecord, and/or forwarded to other User Devices—usually Participants—whomay each effect the change(s) to their own YouTube Player playbackstate.

The YouTube Player may, in certain embodiments, additionally beconfigured to allow non-Controlling User Devices to effect changes inlocal playback state separately from the global data model of theControlling User Device. Playback actions initiated by non-ControllingUser Devices may be sent to Coordinating Server where they may becaptured as Session Events and added to Session Record. The events maynot effect changes in local playback state for other User Devices.

As each User Device reaches a predetermined timepoint in playbackusually the end of the video—the User Device-side YouTube Player App maysend an event to Coordinating Server where it may be captured as aSession Event and added to Session Record. The Server-side YouTubePlayer App may update global data model to reflect the playback statechange for the originating User Device. The change in global data modelmay then be sent as event(s) to User Device(s)—usually only theControlling User Device—and the User Device-side YouTube Player App forthe receiving User Device may update the local display device inresponse to the event, for example to render a widget showing thecompletion status of all User Devices.

Quiz App

Another example of an Interaction App is Quiz, which allows one UserDevice, which may be called the Quiz Master—usually the SP (Host)—topose an interactive question for one or more (or all) other User Devicesin the Session, usually Participants. The Quiz Master may be able toview answers and summary statistics from the other User Devices. TheQuiz Master may add an instance of the Quiz App to a Canvas,parameterized by question(s), other properties like media, and/or listsof answers for questions where appropriate. Quiz Master may send to aCoordinating Server a representation of new Quiz instance and anychanges to Canvas properties that resulted from including Quiz inlayout. Coordinating Server may capture the event as a Session Event,and may add it to Session Record, update global data model, and/orforward events to other User Devices which may then update theirCanvases and display devices.

The Server-side component of Quiz App maintains a data model of Quizstate for User Devices. As User Devices initiate actions to progressthrough Quiz, the User Device-side component of Quiz App sends theactions to a Coordinating Server, which captures them as Session Eventsand adds to Session Record. Server-side Quiz App updates the Quiz datamodel and forwards events to other User Devices—usually the Quiz Masteronly. The Quiz Master's User Device-side Quiz App may choose to renderQuiz data model to a display device in one of multiple ways. Quiz Mastermight wish to view progress of other User Devices through Quiz, possiblyshowing correct/incorrect answers. Or, Quiz Master might wish to view aspecific User Device's view of Quiz in order to, for example, provideassistance or assess lack of progress.

An important aspect of this method and apparatus is that User Devicesmay be restricted to access only portions of the data model for whichthe Server-side Quiz App has authorized them. One user's answers to Quizmight be sensitive, personal, or private data that users other than theQuiz Master may not be authorized to access. Additionally, differentclients may be given different quizzes, which may be personalized by theQuiz Master based on age, gender, experience, skill level, or othersuitable parameter.

Whiteboard App

Another example of an Interaction App is Whiteboard, which allows one ormore User Devices, called Annotators—usually the SP (Host)—to “draw” ona portion of the Canvas. The tools for drawing may include, for example,freehand lines, straight lines from one point to another, shapes likerectangles and circles, solid fill, text labels, eraser, and more. Inbasic usage, Whiteboard may render to display devices with an opaquewhite background, for example, so that Annotators draw on a blank area.

In more advanced usage, Whiteboard may be composed with other Apps by“nesting” them within Whiteboard. This allows Annotators to draw “on topof” other Apps without the other Apps needing to be aware of Whiteboardfunctionality.

A User Device composing a Canvas—usually associated with the SP(Host)—may add an instance of Whiteboard App to a Canvas, parameterizedby (i) background, or App nested within it; and/or (ii) whether all UserDevices share the same Whiteboard state or whether each User Device hastheir own state. Composing User Device may then send to a CoordinatingServer a representation of new Whiteboard instance and any changes toCanvas properties that resulted from including Whiteboard in layout.Coordinating Server may capture the event as a Session Event and may addit to Session Record, update global data model, and/or forward events toother User Devices which may update their Canvases and/or displaydevices.

Initially, Whiteboard may be in an inactive state with no active drawingtools. Any actions initiated from a User Device's input device thattarget the Canvas area of Whiteboard, for example a “mouse click”, may,in certain configurations, be forwarded to an App nested withinWhiteboard.

Annotator may select and activate a drawing tool by an explicit actioninitiated from input device. The action may be sent to a CoordinatingServer, captured as Session Event, and added to Session Record.Properties of the drawing tool such as line width, color, etc., may bealtered by further actions. Whiteboard may now be active and actionstargeting the Canvas area of Whiteboard may be interpreted by the activedrawing tool, for example by rendering a line on the display device.Each drawing action may be sent to a Coordinating Server, captured as aSession Event, and added to Session Record, updating global data model.If Whiteboard was parameterized to share state across all User Devices,then events may also be forwarded to other User Devices which may thenrender the result of drawing action to display device.

Annotator may then initiate an action from input device to deactivateWhiteboard, in which no drawing tools are active and Whiteboard returnsto its initial state.

Attention App

Another example of an Interaction App is Attention, which may allow afirst User Device, denoted Requester—usually a Participant—to requestthat a second User Device, denoted Observer—usually the SP(Host)—observe Requester's Canvas in order to provide live feedback.

Attention App may be (i) added to Canvas; (ii) a representation ofAttention App may be sent to a Coordinating Server along with anychanges to Canvas properties that resulted from including Attention inlayout; (iii) Coordinating Server may capture the event as a SessionEvent and add to Session Record, updating global data model; and/or (iv)Coordinating Server may forward events to other User Devices which mayupdate their Canvases and display devices.

The User Device-side component of Attention App may initially render toObserver's display device as a button labeled “Enable Attention”, forexample. Other User Devices' display devices might have a disabled,“grayed-out” button rendered that's labeled “Request Attention”, forexample.

Observer may activate Attention through an action initiated by an inputdevice, for example a “mouse click”. The initiating action may be (i)sent to a Coordinating Server; (ii) captured as a Session Event andadded to Session Record; (iii) used to update global data model; (iv)and/or forwarded to other User Devices. Other User Devices—the potentialRequesters—may then have Attention render to their display device anactive, non-“grayed out” button labeled “Request Attention”, forexample.

Observer may see rendered to his or her display device a full-canvasview of other users associated with other User Devices. Therepresentation of other users might be chosen based on operatingenvironment such as processor, network interface, output display device,etc. Representations of other users might be “avatars” with text labelsof user's names; or views of other User Devices' Canvases, possibly withreduced rate of Frame updates or simplified views with fewer features;or other suitable views.

A Requester might begin an Attention request through an action initiatedby an input device, for example a “mouse click”. The initiating actionmay be (i) sent to a Coordinating Server; (ii) captured as a SessionEvent and added to Session Record; (iii) used to update global datamodel; (iv) and/or forwarded to only the Observer. The Requester's UserDevice-side Attention App may then render a disabled, “grayed-out”button labeled “Attention Requested”, for example.

The Observer's User Device-side Attention App may render an indicator todisplay device that attention was requested by Requester. This may takemany forms, for example a “badge” rendered over representation ofRequester User Device, or a box containing a list of one or moreRequestors. User Device-side Attention App may render an indication ofthe chronological order of attention requests, for example by reorderingthe Requester and other User Device representations in some way, e.g.,to give priority to a most recent requester that was not yet addressed.

Observer may begin giving attention to a Requester through some actioninitiated by an input device, for example a “mouse click”. Theinitiating action may be (i) sent to a Coordinating Server; (ii)captured as a Session Event and added to Session Record; and/or (iii)used to update global data model. The User Device may then become theActive Requester. The Server-side Attention App may begin sending eventsfrom Active Requester to Observer. This may allow User Device-sideAttention App to render to Observer's display device a view of ActiveRequester's Canvas. Observer can then control Active Requester's Canvasas if Observer were the Active Requester. In other embodiments, Observermay respond to a Request for attention by opening a line ofcommunication with the requester, e.g., a text, audio, and/or videocommunication.

A representation of Observer may also be shared to Active Requester.This representation might be an avatar with text label of Observer'sname; a live stream from Observer; or some other. The User Device-sideAttention App for Active Requester may render the Observer'srepresentation to display device. This representation might be renderedas a “picture in picture” box in a corner of Canvas, for example.

Observer and/or Active Requester may choose to end attention through anaction initiated by an input device, for example a “mouse click”. Theaction may be performed by either the Observer or the Active Requester,or by both the Observer and the Active Requester. Actions may, forexample, be initiated simultaneously by Observer and Active Requester;the actions may be serialized by a Coordinating Server into SessionRecord, and global data model resolved into consistent state. Thedisplays for both Observer and now-former Active Requester may update tostate similar to before Requester became the Active Requester.Observer's display may additionally update to show now-former ActiveRequester without a badge or other previous indicator, and potentiallyreordered within list of User Devices.

Observers may, in certain embodiments, set limits on the number ofattention requests from any one User Device or overall from all UserDevices during a Session. In turn, a Requester that has exhausted his orher quota of requests might initiate some action to be allocatedadditional requests, including some form of payment to Observer.

Instant Replay App

Another example of an Interaction App is Instant Replay, which may allowa User Device to record and/or annotate a Canvas or a portion thereof.Instant Replay may build on the functionality of other apps, and mayoften be used with the Whiteboard and Attention Apps as described below.In some cases, Instant Replay App may allow a user—usually aParticipant—to specify which one or more of the Interaction Apps in theuser's Canvas to record and/or annotate.

Instant Replay App may be (i) added to Canvas, and may be parameterizedby whether recording is automatic or manual; (ii) a representation ofInstant Replay App may be sent to a Coordinating Server along with anychanges to Canvas properties that resulted from including Instant Replayin layout; (iii) Coordinating Server may capture the event as a SessionEvent and add to Session Record, updating global data model; and/or (iv)Coordinating Server may forward events to other User Devices which thenupdate their Canvases and display devices.

In manual recording mode, the User Device-side component of InstantReplay App may render to display device a button or indicator, e.g., anunobtrusive green or red circle that indicates the ability to startrecording Canvas content or a portion thereof. User Device may beginrecording an Instant Replay through some action initiated by an inputdevice, for example a “mouse click”. The event may be sent to aCoordinating Server, where it may be captured as a Session Event andadded to Session Record, updating global data model. In response totransitioning into recording mode, User Device-side Instant Replay Appmay render a new button or indicator, e.g., a red or black square toindicate the ability to stop recording.

In manual recording mode, Instant Replay may record until a stoppingevent occurs. A stopping event may include (i) action of a userexplicitly indicating to stop; (ii) Instant Replay App removed fromCanvas; and/or (iii) current Canvas replaced with new Canvas. Any suchstopping event may be captured as a Session Event by Coordinating Serverand added to Session Record.

A recording created by Instant Replay may contain similar data as aSession Record, and in fact might, in certain embodiments, beimplemented as a nested instance of a Session Record, or as a referenceto a portion of the Master Session Record. The Instant Replay recordingmight be partially or entirely buffered on the User Device, or bufferedentirely on a Coordinating Server. User Devices and Coordinating Serversnegotiate optimal division of responsibilities based on User Devicecapabilities such as processor(s), memory, presence or absence ofstorage device, network interface, geographical location, or othersuitable factors.

When a User Device explicitly stops recording through an actioninitiated by an input device, for example a “mouse click”, the event maybe sent to a Coordinating Server. Coordinating Server may bundle theaction and a reference to the Instant Replay recording as a SessionEvent and add to Session Record. Global data model may also be updated.

In automatic recording mode, and after a recording is stopped in manualmode, User Device-side Instant Replay App may render to display device:(i) playback controls like play, pause, skip 15/30 seconds forwards orbackwards, adjust playback speed to 0.5×, 1.0×, 1.5×, etc; (ii) clearindicator that the rendered Canvas (or portion thereof) content is fromInstant Replay recording as opposed to live. In some embodiments, asplit screen may be utilized to show the live session simultaneouslywith the recorded portion. Further customization of behavior formanual-recording mode might include first rendering a “paused” view ofthe first timepoint in Instant Replay recording under playback, andanother control like a button that ends the playback mode of InstantReplay and returns to the initial mode.

Instant Replay App may often be “nested” within an annotation App likeWhiteboard. This allows Annotators to draw over content under InstantReplay.

Actions by User Device to affect playback state of the Instant Replayrecording—for example, pausing, advancing, etc.—may be sent to aCoordinating Server and captured as Session Events in the SessionRecord. Actions by the User Device to annotate Instant Replay recordingby means of the annotation App's tools may be processed similarly.

For manual-recording mode, User Device may end playback through anaction initiated by an input device, for example a “mouse click”. Thisaction may be sent to a Coordinating Server, captured as a SessionEvent, and added to Session Record. The User Device-side Instant Replayapp may return to rendering the live view of Canvas, or portion thereof,as it was rendering initially before recording was started.

Similar to the YouTube Player App, the User Device that instantiated theInstant Replay App—usually associated with the SP (Host)—may choosewhether or not other User Devices'—usually Participants'—playback statemay diverge from instantiating User Device's. When divergence isallowed, User Devices may progress through Instant Replay recordings attheir own pace. They may also re-sync to the instantiating User Device'sview of the Instant Replay App.

A User Device composing a Canvas—usually associated with the SP(Host)—can nest an instance of Instant Replay App inside Whiteboard Appand/or inside Attention App to allow the Attention Observer to annotateInstant Replay recordings created by either the Attention Observer orthe Attention Requester. When Observer grants attention to a Requester,then Observer and Requester may “share” the same Instant Replay controlof the Requester's Canvas (or portion thereof). Thus, either Observer orRequester can start, stop, and annotate Instant Replay recordings.

In certain circumstances, many individual non-Observer User Devices mayindependently create Instant Replay recordings, of their own Canvases orportions thereof, before requesting attention on a recording they wishto be annotated by Observer. After creating the recording the UserDevice may request attention.

Alternatively, a user composing a Canvas—usually SP (Host)—might chooseinstead to nest Attention App inside Instant Replay App. In thisconfiguration, attention Observer may have the ability to create InstantReplay recordings of their own Canvas independently of the state ofattention grants in Attention App.

Small Group App

Another example of an Interaction App is Small Group, which allows auser (e.g., via the User Device associated therewith)—usually the SP(Host)—to partition other users—usually Participants—into smaller groupsin which each group member shares a single Canvas in the manner of anAttention Observer and Requester in the Attention App described above.In Small Group, one or more users may share the same Canvas.

User Device may, in some embodiments, be partitioned automatically bytarget group size—for example, into groups of at most three UserDevices. Alternatively, Controlling User Device may choose to manuallypartition User Devices. In automatic partitioning, User Devices might berandomly assigned to groups, or additional metadata about “useraffinity” might be recorded pre-session to constrain the automaticpartitioning. For example, User Device metadata could be created thatprevents User Device A from sharing a group with User Device B, or suchthat User Device C always shares a group with User Device D, forexample. If automatic partitioning fails due to unsolvable constraints,Controlling User Device may be required to manually partition somesubset of User Devices. As another example, metadata relating to skilllevel, experience, performance metrics (e.g., score on a quizadministered during the session), or any other suitable data about aUser Device or user may be used as a factor in automatic partitioning.

User Device-side component of Small Group App may wish to renderrepresentations of sharing User Devices to display device. For example,a “picture-in-picture”box might be shown with avatars and text labelsfor each sharing User Device, live streams from User Devices' primaryvideo input devices may be shown, or some other suitable representationmight be shown.

Actions initiated by any of the sharing User Devices may be sent to aCoordinating Server, captured as Session Events and added to SessionRecord, and global data model updated accordingly—including to theshared Canvas portion of the data model. Any further events, includingnotifications of updates to the shared Canvas portion of the global datamodel, may be sent to all sharing User Devices.

Actions may be initiated simultaneously by multiple User Devices sharinga Canvas. Server-side component of Small Group App may serialize actionsby way of Session Events added to Session Record, and resolve globaldata model into consistent state.

Small Group App may be composed with other Apps, for example by nestingan instance of Attention App inside an instance of Small Group App. Inthis configuration, any of the User Devices sharing Canvas in SmallGroup App may request attention of Observer. And when Observer grantsattention, Observer sees shared Canvas of sharing User Devices. Andsharing User Devices see a representation of Observer, for example as a“picture-in-picture” box as described in Attention App.

Additionally, the User Device-side components of other Apps may beconfigured to be aware of Small Group App, while other Apps may not needto be aware of Small Group App. For example, Quiz App may have a singlecanonical data model and representation that all User Device-sidecomponents of sharing User Devices can render without ambiguity. QuizApp therefore may not need to be aware of Small Group App.

However, it may be advantageous for Live Stream App to be aware of SmallGroup App. If an instance of Live Stream App is composed onto Canvaswithin enclosing Small Group App and parameterized to stream primaryvideo input source, which video source—from which sharing User Device—isselected as primary? Live Stream App may wish to expose an additionalcontrol to all sharing User Devices in this situation, allowing anysharing User Device to choose the Live Stream source. (Even in casesoutside of Small Group App, there might be ambiguity about which inputvideo sources from a single User Device is “primary” or “secondary” andso on, so such a control might see general use outside of Small GroupApp.) Live Stream App may also choose a default adaptive policy ofshowing an input video stream from the “dominant speaker” moment bymoment.

Post session: Replaying Sessions

After live Sessions, users may wish to later play back the recording ofthe Session, possibly multiple times. To play back a Session, UserDevice may first send the request to a Coordinating Server. CoordinatingServer may then locate a Session Record on storage device and initiateplayback.

At a high level, playback may be implemented by Coordinating Serverreading the sequence of Session Events from Session Record,re-dispatching Session Events to Server-side components of Apps,updating global data model in return, and/or sending events—includingchanges to User Device's view of global data model—to User Device muchlike during live Sessions. User Device may dispatch events to UserDevice-side components of Apps to effect the same rendering and otherupdates as during live Session.

Replay of a Session may be implemented simply as an instance of InstantReplay App specially locked into annotation mode as described above inInstant Replay App. Instead of referencing an Instant Replay recordingcreated by Instant Replay App in recording mode, this special instanceof Instant Replay might instead reference a master Session Record.Otherwise, all other annotation tools and playback controls may be used.

To navigate a Session Record, playback User Devices may useautomatically-constructed indexes of important events from the SessionRecord. One such index might be represented as a hierarchical table ofcontents, in which the sequence of contents is derived from thechronology of the Session Record, and the hierarchy is derived from thecomposition of Apps within Canvases or other structural information. Anexample table of contents might look like the following:

-   -   1. [Title of first Canvas]        -   a. Playback of YouTube video X starts        -   b. Playback of X seeks to time 10:14.37        -   c. Playback of X ends    -   2. [Title of second Canvas]        -   a. Page 1 in PDF Y shown        -   b. Page 2 in PDF Y shown        -   c. Instant Replay recording starts        -   d. Instant Replay recording ends        -   e. Instant Replay annotation added    -   3. [Title of third Canvas]        -   a. Page 3 in PDF Y shown        -   b. Attention requested        -   c. Page 2 in PDF Y shown        -   d. Attention granted        -   e. . . .

Each item in a table of contents may be linked to a timepoint in SessionRecord by virtue of it representing a Session Event. The playback UserDevice may for example, by “clicking” on an item, “seek” theSession-Record's playback state to the time of the indexed SessionEvent. For ease of browsing, hierarchical sections in theautomatically-generated table of contents may be expanded and collapsed.

As replay of a Session may be implemented using a Canvas, similarly to alive Session, User Device may be allowed to add new Apps to replayCanvas that weren't composed during live Session. For example, a UserDevice may wish to overlay a Timer App to limit the amount of time UserDevice spends practicing a section of a song. In addition, User Devicemay change the playback state of certain Apps during replay in such away that playback state temporarily diverges from Session Record. Forexample, while replaying Session, User Device may choose to seek orrepeat a video source replayed by YouTube Player App. Such a divergencemight in general “pause” playback of the Session Record. User Device mayinitiate a “play” action, sent to Coordinating Server, to resumeplayback of Session Record, in which case temporarily-diverged playbackstate of YouTube App re-assumes the state it had during live Session atthat timepoint.

Playback for the SP (Host) User Device may be granted an additionalcapability: a control may allow the SP (Host) to assume the view ofglobal state from any Participant in the Session. However, in general,playback for a Participant may not allow the Participant to assume theview of any other User Device from the Session.

Because replay of Sessions will generally re-use the same tools and Appsas in live Sessions, User Devices might choose to record the playback ofthe original Session Record into another Session Record, including thenew divergences from original Session Record, and other actionsinitiated during playback.

Systems according to aspects of the disclosure thus provide anextensible platform that allows for richer interaction between serviceproviders and their clients. Disclosed systems create an app ecosystemto support content sharing, playback, multi-camera views and variousapplications to enrich the experience. Service providers should be ableto fully run their business through the app. They want an experiencethat is drastically better than in person. SP capabilities may includechoosing apps, choosing what levels of control clients can exercise,access to a dashboard, and practice sessions. Client experiences mayinclude choosing apps, local device optimizations, direct streaming,practice sessions, post-session replay.

Moreover, the recent advent of social distancing has created anopportunity for a first class 1:1 experience for service delivery inareas such as fitness training, music lessons, skill development, andtherapy. The experiences provided by the disclosed systems couldinclude: Pre-session: Simple signup, Scheduling, Payments, and/orPre-work (e.g., questionnaires); In-session: Customized & interactiveexperience, and/or 1:1 group interaction; Post-session: Practiceprograms and reminder, Content repository, Shareable tools, Customerdevelopment, and/or Tracking of progress.

Systems and methods for providing a remote interactive experience areprovided. Systems and methods may include computer executable codestored in a non-transitory memory and run on a processor. The remoteinteractive experience may be provided via a digital platform.

The platform may include a central server. The central server mayinclude the processor, the non-transitory memory, and/or the computerexecutable code. The central server may include a network interface. Thecentral server may be partially or fully cloud based. The central servermay be distributed across multiple physical or logical entities. Thecentral server may be connected, via the network interface, to a hostdevice and/or to a group of one or more participant devices. The centralserver may coordinate some or all communication between the devices. Thedevices may be computing devices, such as laptops, desktops, tablets,smart phones, or any other suitable device.

The platform may receive instructions from a host device to configure avirtual room. The virtual room may be configured to be accessible to thedevices.

The platform may receive, from a first participant device, a request toaccess the virtual room. In response to the request, the platform mayrender a canvas on a screen of the first participant device. The canvasmay include a customizable graphical representation.

The canvas may be configured at least in part based on the instructionsof the host device. The instructions may include a selection of one ormore interaction apps, from a set of interaction apps, for inclusion inthe canvas.

The one or more participant devices may be associated with a level ofaccess. The level of access may be based at least in part on theinstructions. The level of access may include a set of rules. The set ofrules may define, for each of the one or more participant devices, whichinteraction apps are included in the canvas. The set of rules maydefine, for each of the one or more participant devices, whichinteraction apps the participant device is permissioned to activate. Theset of rules may define, for each of the one or more participantdevices, a level of permission of the participant device to modify aninteraction app.

The interaction apps may include a coordinated window. The coordinatedwindow may be activated by the host device or the first participantdevice. The coordinated window may be initiated with a layout and acontent. The layout may include a size, a shape, and a screen position.The content may include words and/or images that may be displayed withinthe coordinated window. The coordinated window may be rendered on thecanvas of the host device, and on the canvas of the first participantdevice, with the same layout and content.

When the host device or the first participant device executes an actionto modify the layout or the content of the coordinated window, themodification may be applied to the coordinated windows displayed on bothdevices.

In some embodiments, the platform may receive requests, from otherparticipant devices, to access the virtual room. In response to therequests, the platform may render a canvas on the screens of the otherparticipant devices. When any one of the devices executes a modificationaction, the modification may be applied to the coordinated windowsdisplayed on all the devices.

In certain embodiments, the content on the coordinated window mayinclude: a pdf file, an image file, a web browser, a shared documentwith a text editor, or any other suitable content.

In some embodiments, the first participant device may only bepermissioned to activate or modify the coordinated window when the hostdevice selects an option to provide the permission.

In certain embodiments, the canvases may be configured to include atleast two interaction apps that are video tiles, and at least oneinteraction app that is a coordinated window. The participants in theremote session may thereby communicate with each other with audiovisualcommunication while simultaneously collaborating on the content in thecoordinated window.

Systems and methods for providing a remote interactive experience areprovided. The systems and methods may include computer executable codestored in a non-transitory memory and run on a processor. The remoteinteractive experience may be provided via a digital platform. Theplatform may provide management of a shared browser. A shared browsermay be otherwise referred to herein as a coordinated window. The sharedbrowser may be shared by a plurality of participant devices. The sharedbrowser may, in certain embodiments, be included as an app in a canvas.In some embodiments, the shared browser may be deployed independently ofa canvas.

The platform may store the shared browser in a cloud-based location. Thecloud-based location may be a central server. The shared browser maythus be cloud-based. The platform may enable the plurality ofparticipant devices to access the cloud-based shared browser. Theplatform may receive, as input, actions performed to each of theplurality of participant devices. The actions may include at least oneof an action list that includes: a cursor movement, a window-sizing, awindow-positioning, and a file navigation, performed to the sharedbrowser. In certain embodiments, the action list may further include aninput, a gesture, a document modification, or any other suitable inputaction, performed to the shared browser. The platform may propagate theactions to the shared browser.

In some embodiments, each of the plurality of participant devices may beassociated with a permission level. The permission level may enable theparticipant device to execute a pre-determined set of actions forpropagation to the shared browser.

In certain embodiments, the platform may be configured to coordinatemultiple inputs performed on a plurality of separate participantdevices. The coordinating may include animating pre-determined actionson the shared browser in order to notify other users that the actionsare being performed on a device. The pre-determined actions may includea clicking action, a dragging action, and/or typing.

The coordinating may include providing an exclusive focus for afirst-performed action. Providing an exclusive focus may includeproviding an exclusive browser focus for a pre-determined time period.Exclusive browser focus may prevent other actions from being applied tothe browser for the pre-determined time period. Other actions mayinclude actions performed to participant devices other than theparticipant device to which the first-performed action was performed.

The coordinating may include, during the pre-determined time period,buffering the other actions in a buffer. The platform may apply thebuffered actions, in an order that the buffered actions were performed,when the pre-determined time period terminates.

The coordinating may include tracking the state of the browser when eachof the other actions was performed. The platform may apply, when thepre-determined time period terminates, the buffered actions, asperformed to the tracked state of the browser, to the current state ofthe browser.

For example, Action 1 on Device 1 may be a first-performed action.Action 1 may, for example, include typing input. Exclusive browser focusmay be provided to Device 1 for a time period. The time period may, forexample, be 1 minute. The time period may, in another example, be untilan event occurs, such as 10 seconds without any typing input from Device1. Device 2 may perform Action 2 during the time period. Subsequently,Device 3 may perform Action 3, also during the time period. Action 2may, for example, include a mouse click. Action 3 may, for example,include a navigation input. When the time period concludes, and theexclusive browser focus is withdrawn, the platform may apply Action 2and then Action 3 to the browser. The mouse click of Action 2 may beapplied to a location to which the click was actually performed, even ifthe location has since moved by the time the action is being applied.Similarly, the navigation of Action 3 may be applied as it would havebeen based on the state of the browser at the time of performance ofAction 3, even if the state of the browser has since changed by the timeof application.

The coordinating may include, when the actions comprise a plurality ofaudio inputs, combining the audio inputs to form a combined audio input.For example, if the shared browser receives audio input, such as a voicecommand or a singing session, and two or more of the devices receiveaudio input through their respective microphones, the platform maycombine the audios into a single audio file to input to the sharedbrowser.

In some embodiments, the platform may be configured to activate securitymeasures to protect sensitive information from being compromised in theshared browser. The activation may be triggered manually. The activationmay, in certain embodiments, be activated automatically. Automaticactivation may be triggered by the platform determining that a portionof the shared browser contains, or is associated with a likelihood ofcontaining, sensitive information.

The security measures may include identifying that the sensitiveinformation is accessible in the shared browser and securing the sharedbrowser. Securing the shared browser may include providing an app-levelfeature that tracks a participant device associated with the sensitiveinformation. When the participant device associated with the sensitiveinformation departs a predetermined location, the platform may terminateaccess of some or all of the participant devices to the sensitiveinformation. The platform may, alternatively or additionally, notify theparticipant device associated with the sensitive information that thesensitive information is vulnerable to being accessed by the other userdevices.

Departing a predetermined location may include a user associated withthe participant device associated with the sensitive informationdeparting a physical location associated with the participant deviceassociated with the sensitive information. Some examples may include theparticipant device associated with the sensitive information sensingthat its user leaves to a location out of camera view of the device,moves a threshold distance away from the device, or moves through adoorway and into another room. Departing a predetermined location may,alternatively or additionally, include the participant device associatedwith the sensitive information logging out of a virtual room throughwhich the participant devices may be accessing the shared browser.

The security measures may also include receiving a selection of aprivacy option on one of the participant devices. The privacy option maybe a default setting. In response to receiving the selection, theplatform may activate the privacy option on the one of the participantdevices. The privacy option may be configured to enable a privateportion of the shared browser. The private portion may be visible and/ormodifiable only on the participant device that selected the option. Theprivacy option may be automatic or manual. For example, the privacyoption may be configured to automatically detect a sensitive portion ofthe browser. Alternatively, a device may be able to request that acertain portion of the browser displayed on its screen be kept private.

The security measures may also include receiving a selection of asecurity option on one of the participant devices. In response toreceiving the selection, the platform may activate the security option.The security option may be configured to prevent actions performed onother participant devices from being applied to the shared browser.

In some embodiments, the security option may be configured to preventcertain actions performed on other participant devices from beingapplied to the shared browser. The certain actions may include actionsto navigate the shared browser away from a page being displayed in theshared browser.

The security measures may also include identifying a sensitive inputfield on the shared browser. The sensitive input field may, for example,be a field for inputting identifying information (e.g., name, address,date of birth, social security number), financial information (e.g.,bank account or credit card information), or other suitable sensitiveinformation. The platform may identify a participant device whichinitiated the shared browser as a host device for the sensitive inputfield. The platform may identify a device associated with the sensitiveinput field as the host, even if that device did not initiate the sharedbrowser. The platform may prevent participant devices other than thehost from inputting information into the sensitive input field and/orviewing information inputted into the sensitive input field.

In some embodiments, the platform may be configured to transfercredentials from a local system to the shared browser. The transfer maybe executed in response to a selected option. The option may, in certainembodiments, be a default option, and may be executed automatically.

In certain embodiments, the platform may be configured to capturebrowsing data on the shared browser. The platform may be furtherconfigured to transfer the browsing data to a local system. The transfermay be executed via a secure channel. The platform may be furtherconfigured to store the browsing data on the local system. The capture,transfer, and/or storage may be executed in response to a selectedoption. The option may, in certain embodiments, be a default option, andmay be executed automatically.

The platform may thus provide for local storage of user data for a webbrowser, such as a shared browser, running on a cloud service. A typicalweb browser may save website data for user authentication and/or forpersisting a state of the website across multiple website visits.

For a browser, such as the shared browser, running in the cloud it maybe desirable to store the data on the user's local computer, instead ofin the cloud. Doing so may give full ownership of the data to the userand may eliminate security concerns about storing user data in thecloud. Using a cloud browser may thereby become as secure as the user'slocal browser.

Websites loaded by a web browser may typically save three types ofdata-cookies, local storage, and session storage. While a browser isrunning in the cloud, the data saved by the browser may be captured andsecurely transmitted on the network to the browser running on the localcomputer that is presenting the remote browser view. The local computercan then save the data locally in persistent storage. Conversely, whenthe cloud browser loads a specific website it can request and receiveover the network the local data saved in persistent storage beforestarting to load the website, making the data available to the web sitewhile loading it.

A web browser may present different programming interfaces to thewebsite for storing cookies and local/session storage. For codeefficiency, ease of development, and reduced maintenance it may bedesirable to present a unified abstracted interface that can be used forpersisting this data. Cookies and local/session storage data may, incertain embodiments, be persisted using some or all of the illustrativeprogramming interface presented, along with explanatory comments, in thefollowing paragraphs.

public open( ): Promise <void>

/*Opens a persistent storage instance and prepares it for operations. Ona first initialization of the persistent storage, it may create thestores for local storage, session storage, and cookies.*/

public close( )

/*Closes the persistent storage instance.*/

public write(

storeName: PersistentStorageStoreName,

origin: string,

dataKey: string,

dataValue: string|object,

): Promise <void>

/*Writes the key/value pair for a specific website origin into a namedstore.*/

public delete(storeName: PersistentStorageStoreName, origin: string,dataKey: string): Promise <void>

/*Deletes the key/value pair for a specific website origin from a namedstore.*/

public clear(storeName: PersistentStorageStoreName, origin: string):Promise <void>

/*Clears all data for a specific website origin from a named store.*/

public readAll(storeName: PersistentStorageStoreName, origin?: string):Promise <object[ ]>

/*Reads all the data for a specific origin from a named store.*/

public cleanup( )

/*Removes all stored data from the persistent storage.*/

This interface may, in certain illustrative embodiments, be implementedin a modern web browser using a tool such as IndexedDB which may provideasynchronous data storage and retrieval. Since IndexedDB allows queryingusing a specific key, it may allow fast retrieval of the data for aspecific website origin, when the cloud browser requests such data whileloading a website.

In some embodiments, the platform may be configured to provide aselectable mechanism, such as a button or icon, on a display of one ofthe participant devices. Selecting the mechanism may trigger a launch ofthe shared browser. Selecting the mechanism may trigger transmission ofan invitation to other participant devices to access the shared browser.

The mechanism may be embedded in the shared browser. In someembodiments, the mechanism may be embedded in a web-based applicationthat is independent of the shared browser. For example, a website thatwants visitors to be able to collaborate (e.g., an artistic or travelbooking website) may provide a mechanism for visitors to initiate ashared browser to collaborate on the website with other participants. Instill other embodiments, the mechanism may be an independent applicationinstalled on a participant device. For example, a smart phone, tablet,laptop, or desktop may have a shared browser application installed. Theshared browser application may include an icon on a homepage of thedevice. Selecting the icon may launch the shared browser. Launching theshared browser may prompt the user for a selection of a content todisplay in the shared browser. Launching the shared browser may promptthe user for a selection of other participant devices to invite to theshared browser.

Apparatus and methods described herein are illustrative. Apparatus andmethods in accordance with this disclosure will now be described inconnection with the figures, which form a part hereof. The figures showillustrative features of apparatus and method steps in accordance withthe principles of this disclosure. It is understood that otherembodiments may be utilized, and that structural, functional, andprocedural modifications may be made without departing from the scopeand spirit of the present disclosure.

FIG. 1 shows an illustrative block diagram of system 100 that includescomputer 101. Computer 101 may alternatively be referred to herein as a“server” or a “computing device.” Computer 101 may be a desktop, laptop,tablet, smart phone, or any other suitable computing device. Elements ofsystem 100, including computer 101, may be used to implement variousaspects of the systems and methods disclosed herein.

Computer 101 may have a processor 103 for controlling the operation ofthe device and its associated components, and may include RAM 105, ROM107, input/output module 109, and a memory 115. The processor 103 mayalso execute all software running on the computer—e.g., the operatingsystem and/or voice recognition software. Other components commonly usedfor computers, such as EEPROM or Flash memory or any other suitablecomponents, may also be part of the computer 101.

The memory 115 may be comprised of any suitable permanent storagetechnology—e.g., a hard drive. The memory 115 may store softwareincluding the operating system 117 and application(s) 119 along with anydata 111 needed for the operation of the system 100. Memory 115 may alsostore videos, text, and/or audio assistance files. The videos, text,and/or audio assistance files may also be stored in cache memory, or anyother suitable memory. Alternatively, some or all of computer executableinstructions (alternatively referred to as “code”) may be embodied inhardware or firmware (not shown). The computer 101 may execute theinstructions embodied by the software to perform various functions.

Input/output (“I/O”) module may include connectivity to a microphone,keyboard, touch screen, mouse, and/or stylus through which a user ofcomputer 101 may provide input. The input may include input relating tocursor movement. The input may relate to a remote educationalexperience. The input/output module may also include one or morespeakers for providing audio output and a video display device forproviding textual, audio, audiovisual, and/or graphical output. Theinput and output may be related to computer application functionality.

System 100 may be connected to other systems via a local area network(LAN) interface 113.

System 100 may operate in a networked environment supporting connectionsto one or more remote computers, such as terminals 141 and 151.Terminals 141 and 151 may be personal computers or servers that includemany or all of the elements described above relative to system 100. Thenetwork connections depicted in FIG. 1 include a local area network(LAN) 125 and a wide area network (WAN) 129, but may also include othernetworks. When used in a LAN networking environment, computer 101 isconnected to LAN 125 through a LAN interface or adapter 113. When usedin a WAN networking environment, computer 101 may include a modem 127 orother means for establishing communications over WAN 129, such asInternet 131.

It will be appreciated that the network connections shown areillustrative and other means of establishing a communications linkbetween computers may be used. The existence of various well-knownprotocols such as TCP/IP, Ethernet, FTP, HTTP and the like is presumed,and the system can be operated in a client-server configuration topermit a user to retrieve web pages from a web-based server.

The web-based server may transmit data to any other suitable computersystem. The web-based server may also send computer-readableinstructions, together with the data, to any suitable computer system.The computer-readable instructions may be to store the data in cachememory, the hard drive, secondary memory, or any other suitable memory.The transmission of the data together with computer-readableinstructions may enable the computer system to quickly retrieve thedata, when needed. Because the computer system is able to quicklyretrieve the data, the web-based server may not need to stream the datato the computer system. This may be beneficial for the computer system,because the retrieval may be faster than data-streaming. Conventionally,streaming data requires heavy usage of the processor and the cachememory. If at least some data is stored in the computer system's memory,retrieval of the data may not require heavy processor and cache memoryusage. Any of various conventional web browsers can be used to displayand manipulate retrieved data on web pages.

Additionally, application program(s) 119, which may be used by computer101, may include computer executable instructions for invoking userfunctionality related to communication, such as e-mail, Short MessageService (SMS), and voice input and speech recognition applications.Application program(s) 119 (which may be alternatively referred toherein as “plugins,” “applications,” or “apps”) may include computerexecutable instructions for invoking user functionality relatedperforming various tasks. The various tasks may be related to digitaleducational platforms.

Computer 101 and/or terminals 141 and 151 may also be devices includingvarious other components, such as a battery, speaker, and/or antennas(not shown).

Terminal 151 and/or terminal 141 may be portable devices such as alaptop, cell phone, Blackberry™, tablet, smartphone, or any othersuitable device for receiving, storing, transmitting and/or displayingrelevant information. Terminals 151 and/or terminal 141 may be otherdevices. These devices may be identical to system 100 or different. Thedifferences may be related to hardware components and/or softwarecomponents.

Any information described above in connection with database 111, and anyother suitable information, may be stored in memory 115. One or more ofapplications 119 may include one or more algorithms that may be used toimplement features of the disclosure, and/or any other suitable tasks.

The invention may be operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well-known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, tablets, mobile phones, smart phones and/or otherpersonal digital assistants (“PDAs”), multiprocessor systems,microprocessor-based systems, set top boxes, programmable consumerelectronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, and the like.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc., that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

FIG. 2 shows illustrative apparatus 200 that may be configured inaccordance with the principles of the disclosure. Apparatus 200 may be acomputing machine. Apparatus 200 may include one or more features of theapparatus shown in FIG. 1. Apparatus 200 may include chip module 202,which may include one or more integrated circuits, and which may includelogic configured to perform any other suitable logical operations.

Apparatus 200 may include one or more of the following components: I/Ocircuitry 204, which may include a transmitter device and a receiverdevice and may interface with fiber optic cable, coaxial cable,telephone lines, wireless devices, PHY layer hardware, a keypad/displaycontrol device or any other suitable media or devices; peripheraldevices 206, which may include counter timers, real-time timers,power-on reset generators or any other suitable peripheral devices;logical processing device 208, which may compute data structuralinformation and structural parameters of the data; and machine-readablememory 210.

Machine-readable memory 210 may be configured to store inmachine-readable data structures: machine executable instructions (whichmay be alternatively referred to herein as “computer code”),applications, signals, and/or any other suitable information or datastructures.

Components 202, 204, 206, 208 and 210 may be coupled together by asystem bus or other interconnections 212 and may be present on one ormore circuit boards such as 220. In some embodiments, the components maybe integrated into a single chip. The chip may be silicon-based.

FIG. 3 shows illustrative screenshot 300 of exemplary canvas 301 of aninteractive platform according to aspects of the disclosure. Canvas 301may include background 303. Background 303 may be selected by the hostas a background for some or all of the canvases in a session. Background303 may be a default background. The default background may, in certainembodiments, be automatically selected by the platform based on asession category. For example, the session shown in illustrativescreenshot 300 may be a group guitar lesson, and background 303 may bean image of a person playing a guitar. A title may be displayed (shownin screenshot 300 above background 303) which may describe a particularcanvas and/or session.

Canvas 301 may include participant icons 305-315. Participant icons305-315 may include text, avatars, images, or other suitable elementsfor representing and/or identifying some or all of the participants in asession. Participant icons 305-315 may include video streams of theparticipants. The video-streams may communicate a live video stream fromparticipant devices. In the exemplary session shown in screenshot 300,participant icons 305-315 may represent video feeds of six participantsin the session, who may be six students attending a group guitar lesson.In certain embodiments, one of participant icons 305-315 may identify,represent, and/or show the session host.

Tiles 317-327 may be selectable elements for use in configuring canvas301 prior to the session and/or during the session. Tile 317 mayrepresent an option to find additional apps. Tiles 319-327 may representapps already selected and/or available for use in conjunction withcanvas 301 as part of the session.

Tiles 317-327 are shown displayed in a column at the left of screenshot300, and may, in other embodiments, be displayed in other suitableorganizational layouts. The layout of tiles 317-327, and some or all ofthe other elements shown in screenshot 300, may have certain defaultpositionings, and some or all of the layouts may be customizable by thehost and/or other user. Customizations may include sizing, ordering,positioning, colors, borders, backgrounds, etc.

Screenshot 300 may represent the view of the host or other userconfiguring the canvases in a session. The view from a participantdevice may differ from the view of the host. For example, a canvasrendered on a participant device may not include any of tiles 317-327. Acanvas rendered on a participant device may have only some of tiles317-327 (e.g., communication apps or apps such as record, annotate,request attention, etc.). Furthermore, each participant device may showall participant icons 305-315, or, in some embodiments, may show onlythe participants aside from the participant associated with thatparticular device. In some embodiments, each participant device may showonly the participant associated with that device. Some or all of theexemplary differences between a host view and a participant viewdescribed above, may be customizable by the host and/or theparticipants.

FIG. 4 shows illustrative screenshot 400 of exemplary canvas 401 of aninteractive platform according to aspects of the disclosure. Canvas 401may show further features of a canvas similar to canvas 301. Participanticons 405-415 may correspond to participant icons 305-315 shown in FIG.3. Tiles 417-427 may correspond to tiles 317-327 shown in FIG. 3.

App menu 403 may present additional apps available for inclusion in thecanvas. App menu 403 may be displayed as a result of selecting tile 417.Selecting an app from app menu 403 may include a mouse click and/ordrag. App menu 403 may include an option to add more apps (shown inscreenshot 400 at the top right corner of app menu 403). Selecting theoption to add more apps may scroll down the app menu, link a user to anapp store, or provide tools for the user to design a new app.

FIG. 5 shows illustrative screenshot 500 of exemplary canvas 501 of aninteractive platform according to aspects of the disclosure. Canvas 501may show further features of a canvas similar to canvas 301. Participanticons 505-515 may correspond to participant icons 305-315 shown in FIG.3. Tiles 517-527 may correspond to tiles 317-327 shown in FIG. 3.

Window 503 may show a video clip or other presentation that the hostwants to play for the participants. Window 503 may correspond to the appavailable via tile 521 in the column at the left of screenshot 500.Canvas 501 may, in certain embodiments, have been pre-configured by thehost prior to the session to include window 503. Canvas 501 may be thefirst canvas in a series of canvases configured by the host for thesession. In some embodiments, the host may modify, in real-time, acanvas currently running during a session to include one or more appssuch as window 503.

In some embodiments, the presentation of window 503 may be stored on thehost device, and the host device may transmit the presentation to one ormore of the participant devices for viewing. In other embodiments, thepresentation may be stored on a separate server (e.g., accessible atYouTube.com), and the platform may direct the participant devices toaccess the presentation directly from the separate server. For example,window 503 shown in screenshot 501 may be a video clip of a guitarperformance that the host (e.g., a guitar instructor) may wish for theparticipants (e.g., guitar students) to view during the session (e.g.,the group guitar lesson). The video clip may be accessible on YouTube,and the platform may coordinate for the participant devices to accessthe clip from YouTube at a particular time during the session asdirected (in real-time or pre-session) by the host.

FIG. 6 shows illustrative screenshot 600 of exemplary canvas 601 of aninteractive platform according to aspects of the disclosure. Canvas 601may show further features of a canvas similar to canvas 301. Participanticons 605-615 may correspond to participant icons 305-315 shown in FIG.3. Tiles 617-627 may correspond to tiles 317-327 shown in FIG. 3. Canvas601 may be a new canvas, e.g., a canvas configured prior to a session aspart of a series of canvases. For example, canvas 501 may be a firstcanvas and canvas 601 may be a second canvas.

Canvas 601 may include windows 603, 629, and 631. Window 603 may showmusical notes of a song being taught in a session. Window 603 maycorrespond to the app available via tile 623 in the column at the leftof screenshot 600. Window 629 may show a camera feed (live, or, incertain embodiments, pre-recorded) of the host of the session. Window629 may correspond to the app available via tile 619 in the column atthe left of screenshot 600.

Window 631 may show a second camera feed of the host of the session. Thecamera feed shown in window 631 may be a zoomed-in version of the camerafeed shown in window 629, or may show footage from a different camera.Alternatively, the footage shown in window 631 may include pre-recordeddiagrams, images, or video. The footage shown in window 631 may, in theparticular illustrative example shown in FIG. 6, show a close-up view offinger placements on a guitar, which may be part of an educationalexperience of the session. Window 631 may correspond to the appavailable via tile 627 in the column at the left of screenshot 600.

Windows 603, 629, and 631 may have been pre-configured to be included incanvas 601 by the host prior to the session. Alternatively, some or allof windows 603, 629, and 631 may be selected during a session forinclusion in an existing canvas, e.g., to modify canvas 501 and replacewindow 503.

FIG. 7 shows illustrative screenshot 700 of exemplary canvas 701 of aninteractive platform according to aspects of the disclosure. Canvas 701may show further features of a canvas similar to canvas 301. Participanticons 705-715 may correspond to participant icons 305-315 shown in FIG.3. Tiles 717-727 may correspond to tiles 317-327 shown in FIG. 3. Canvas701 may be a new canvas, e.g., a canvas configured prior to a session aspart of a series of canvases. For example, canvas 701 may be a thirdcanvas preset to be displayed after canvases 501 and/or 601.Alternatively, canvas 701 may be configured during a session as amodification of an existing canvas, such as canvas 601 (e.g., via anaction to replace the app running in window 631 with the one running inwindow 731).

Windows 703 and 729 may be substantially the same as windows 603 and 629shown in FIG. 6. Window 731 may present an additional musical teachingtool, such as a metronome, or other suitable tool, which may correspondto the app available via tile 725 in the column at the left ofscreenshot 700.

FIG. 8 shows illustrative screenshot 800 of exemplary canvas 801 of aninteractive platform according to aspects of the disclosure. Canvas 801may show further features of a canvas similar to canvas 301. Participanticons 805-815 may correspond to participant icons 305-315 shown in FIG.3. Tile 817 may correspond to tile 317 shown in FIG. 3. Tiles 819 and821 may correspond to tiles available in addition to tiles 317-327 shownin FIG. 3. Access to tiles 819 and 821 may be available by selectingtile 817, scrolling down the app column, or some other suitable way ofaccessing additional tiles which correspond to additional availableapps.

Canvas 801 may be a new canvas, e.g., a canvas configured prior to asession as part of a series of canvases. For example, canvas 801 may bea fourth canvas preset to be displayed after canvases 501, 601, and/or701. Alternatively, canvas 801 may be configured during a session as amodification of an existing canvas, such as canvas 701 (e.g., via anaction to replace the apps running in windows 703, 729, and 731 with theone running in window 803).

Window 803 shows a poll that has been included in canvas 801. The pollshown in window 803 may correspond to the app available via tile 821 inthe column at the left of screenshot 800. The poll may offer choices,such as choices 823, 825, and 827. The choices may be options for futureportions of the current session, for future sessions, or any othersuitable options that a host may want to query the participants of thesession. In the example shown in screenshot 800, window 803 may presentoptions for a song for a next lesson. The next lesson may be a futuresession, and the host may use the results in configuring canvases forthe next session.

Results of the poll shown in window 803 may include result icons829-837. Result icons 829-837 may correspond to participants associatedwith participant icons 805-815, and may be positioned next to the polloption a participant voted for. Window 803 may show that result icons829 and 831 (which may correspond to participant icons 805 and 811,respectively) voted for option 823, result icons 833-837 (which maycorrespond to participant icons 807, 809, and 813, respectively) votedfor option 825, and no participants voted for option 827. Theparticipant associated with participant icon 815 appears not to havevoted in the poll.

Result icons 829-837 may be duplicates of whateveridentifier/representation/video may be presented in participant icons805-815. In some embodiments, result icons 829-837 may be different thatthe identifier/representation/video presented in participant icons805-815. For example, if participant icons 805-815 show video feed ofeach participant, result icons 829-837 may show a name, image, or avatarassociated with each participant.

In some embodiments, poll results may be used for a next lesson whichmay be a later part of the current session. In this scenario, the pollresults may contribute to a decision of which canvas to display next.For example, the host may have prepared multiple canvas options and mayselect one based on the poll results. In some embodiments, the selectionmay be automatic. The host, when configuring the multiple canvases, maybe able to select a setting which maps certain canvases to certain pollresults. Thus, when a certain poll option wins, the canvases mapped tothat winning option may be automatically presented next. Moreover, asetting may be available wherein whichever participant voted for acertain poll option, is presented with the canvas or set of canvasesmapped to that poll option.

FIG. 9 shows illustrative system architecture 900 of an interactiveplatform according to aspects of the disclosure. Architecture 900 showscentral server 901. Central server 901 may include multiple logicaland/or physical servers. Central server 901 may include one or moreprocessors 903. Central server 901 may include one or more memory units905. Central server 901 may include one or more network interfaces 907.Central server 901 may include and/or work in conjunction with acoordinating server. Central server 901 may include and/or work inconjunction with a content delivery server. Central server 901 may, incertain embodiments, be partially or completely cloud-based.

Central server 901 may be connected to multiple User Devices. Theconnection between the servers and the devices may be, at least in part,via a network interface of one or more of the servers and/or devices.

A User Device may be a device which may include a processor, memory,network interface, display mechanism, and/or input/output (I/O)mechanism. Each User Device may be associated with a user who may bepart of a session facilitated by the platform. Each User Device may beable to configure, display, modify, respond to, and otherwise engagewith one or more canvases in an interactive remote session.

One User Device may be host device 909. Host device 909 may beassociated with a host user who may have administrative authority. Thehost user may, in certain embodiments, be a platform subscriber. Thehost may be able to initiate a session and/or configure canvases. Insome embodiments, more than one host device may simultaneously, orsequentially, co-host a session.

User Devices may also include one or more participant devices, shown inarchitecture 900 as devices 911-917. The participant devices may beassociated with participants of the session who are not hosts. In someembodiments, participants may not have authority to initiate a session.Participants may, in certain embodiments, still have authority toconfigure, respond to, interact with, and/or modify one or morecanvases. Participant permissions may, in certain embodiments, becustomizable by the host. The host may be able to customize thepermissions for all the participants at once and/or individually foreach participant separately.

An exemplary session involving a group guitar lesson was detailed abovein the screenshots of FIGS. 3-8. In that example, host device 909 may beassociated with a guitar instructor, and participant devices 911-917 maybe associated with guitar students. The disclosed platform may be usefulfor any other remote session. Other exemplary educational uses mayinclude math, science, literature, language, standardized testpreparation, or any other suitable instruction. The platform may also beuseful for book reading and movie viewing groups, recreational andbusiness meetings, and any other imaginable remote session. Thedisclosed platform may be particularly useful when the session featuresa presentation and/or includes an interaction.

FIG. 10 shows illustrative flowchart 1000 of an exemplary remotemulti-stream interactive session on a platform configured according toaspects of the disclosure. Flowchart 1000 may begin when a hostconfigures one or more canvases at step 1001. Configuring a canvas mayinclude selecting content (e.g., one or more apps) and/or content layouton a canvas template for use in a live session. In some embodiments, theplatform may provide one or more default preconfigured canvases fully orpartially ready for use. In some embodiments, the preconfigured canvasesmay be generated by an artificial intelligence (AI) component of theplatform, and may be based at least in part on historical data that mayinclude prior canvases configured by the host.

The actual live session may be initiated by the host at step 1002.Session initiation may be triggered in real-time by the host,prescheduled by the host to self-initiate at a predetermined time or inresponse to a predetermined event, or any other suitable initiationprocess. Once the session is initiated, the platform may transmit aninitial canvas for display on one or more participant devices, shown inthe exemplary session of architecture 1000 as displaying the canvas onparticipant device 1 (PD1) at step 1003 and participant device 2 (PD2)at step 1005. Other scenarios may include less or more than 2participants. Moreover, the canvas displayed on PD1 at 1003 may bedifferent than the canvas displayed on PD2 at 1005. Furthermore, in someembodiments, the host may enable PD1 to modify the canvas displayed onPD1, and/or enable PD2 to modify the canvas displayed on PD2.

The participants may interact or otherwise engage with the canvas in thesession. For example, a participant may respond to a poll/quiz, generatevideo, audio, and/or text-based content (e.g., as a practice or ascommunication with the host and/or other participants), review and/orannotate a portion of the session, or any other suitable interaction orengagement. In some embodiments, interaction with the canvas may includeuser activity such as mouse clicks, cursor movements, keystrokes and thelike.

The platform may generate utilization data based on the engagementsand/or interactions. The platform may, in certain embodiments, generateat least part of the utilization data separately for each participant.Step 1007 shows utilization data gathered for PD1 and step 1009 showsutilization data gathered for PD2.

At step 1011, the platform may analyze the utilization data. Theanalysis may include artificial intelligence (AI) elements. The analysismay include basic analysis, such as generating a score based on quizperformance. The analysis may also be more advanced. For example, in ascenario where the utilization data includes content created by aparticipant, an AI agent may score the content.

As a more detailed example to illustrate the steps shown in architecture1000, an illustrative session the host may be a language instructorteaching a group of participants a language such as, for example,Mandarin. A canvas displayed on PD1 and PD2 may include instructionalmaterial, such as a video clip of a person saying “Hello” in Mandarin.One or both of the participants may respond to the instructionalmaterial of the canvas by recording an audio or audiovisual clip ofthemselves attempting to practice saying “Hello” in Mandarin. Thepractice attempts may be analyzed via AI agents that may incorporatelanguage processing components. The platform may generate performancemetrics based on the analysis.

At step 1013, the platform may display the performance metrics on adashboard. The dashboard may be displayed on the host device. Thedashboard may be designed with a streamlined, easy to use interface.Some or all dashboard features may be customizable and/or configurable,e.g., by the host. In some embodiments, the dashboard, or a portionthereof, may be displayed on one or more participant devices.

Steps 1015 and 1017 may represent exemplary actions that may be part ofan interactive session. The actions may be activated via one or moreapps that may be included in the canvas. One or more of the actions maybe generalized across all devices, or may, in certain embodiments, beindividualized and directed to one or more devices. For example, step1015 represents an annotation of a playback portion which may only beactivated for PD1. Annotating a playback portion may, in certainembodiments, include an action where the host may annotate a recordedportion of the session, e.g., a recorded practice attempt generated byPD1. The host may, for example, review and annotate the practice attemptbased on metrics displayed in the dashboard.

Step 1017 may represent an action of initiating a sidebar interaction.This action may be directed to one participant, e.g., PD2. The sidebarinteraction may be a result of a request for attention from PD2. Thesidebar interaction may also be initiated by the host in response tometrics displayed on the dashboard. A sidebar interaction may be anindividualized communication with the participant, e.g., a video, audio,and/or text-based communication. The sidebar communication may occur inparallel with the primary session, e.g., via a window positioned at aside or corner of the canvas. The window may only be visible on thecanvases of the host and the participant who is part of the sidebarinteraction.

Further steps of architecture 1000 may include configuring a new canvas,displaying a new canvas, modifying an existing canvas, or other suitablefurther step.

Step 1019 shows that session data may be collected and/orcompartmentalized. Compartmentalized data may include categories, tags,and/or identifiers associated with the data collected. Thecompartmentalized nature of the data may facilitate generatingindividualized post session reports at step 1021. The reports mayinclude an event-based summary of the session. The events included inthe summary may be individualized based on events that are significantfor each participant. The events may be selectable and may enable theuser to easily navigate a recording of the session. For example, in theillustrative session shown in architecture 1000, a post session summarymay be generated for PD1, and the summary may include an event labeled“annotate playback portion.” The user associated with PD1 may be able toselect that event and review the portion of the session recordingcontaining that event.

FIG. 11 shows illustrative system diagram 1100 of an interactiveplatform according to aspects of the disclosure. Diagram 1100 showsparticipant devices 1101, 1103, and 1105. One of the participant devicesmay be a host device. Other embodiments may include fewer, or more,participant devices. There may be no limit on the number of participantdevices that can access a virtual room for a session. In someembodiments, there may be a system cap on the number of participantdevices for security and/or efficiency purposes. In some embodiments,the host device may be able to limit the number of participant devices.

Diagram 1100 shows that the participant devices may be in communicationwith a central server 1107. Central server 1107 may be cloud based.Central server 1107 may be a coordinating server. Central server 1107may receive inputs from the participant devices. Central server 1107 maytransmit instructions to the participant devices. The instructions mayinclude instructions for rendering a canvas on the screens of theparticipant devices. Central server 1107 may, in certain embodiments, beconfigured to coordinate and synchronize the canvases or portionsthereof.

For example, the canvas displayed on the screen of device 1101 mayinclude video tiles 1109-1113 and window 1115. Window 1115 may contain ashared document. Window 1115 may contain editable text. The canvases onthe screens of devices 1103 and 1105 may similarly include correspondingvideo tiles 1117-1121 and 1125-1129, respectively, and also windows 1123and 1131, respectively.

In some embodiments, an action executed on the host device(alternatively, on any participant device that has permission from thehost to do so) to modify window 1115, may be applied to the window onthe canvases of all the devices (e.g., windows 1123 and 1131). Modifyingthe window may include resizing, reshaping, or repositioning the window.Modifying may also include navigating about or cropping the contentwithin the window. These illustrative modifications may be supportedeven when the content within the window is not natively editable. In theexample of editable text, images, or other media, edits to the contentmay also be applied across all the devices. In some embodiments,modifications to the size, shape, and position of video tiles may alsobe generalized across all the devices in the session. In certainembodiments, the host may be able to customize which apps and features,and which types of modifications, are applied across all devices, andwhich can be tailored locally for each participant device.

The platform may thereby provide a coordinated and synchronized toolwhereby participants can communicate with video and audio, whilecollaborating on a shared document or other content. The platform itselfmay augment the document with synchronization capabilities even when thedocument is not natively a shared document.

FIG. 12 shows illustrative flowchart 1200 in accordance with principlesof the disclosure. Flowchart 1200 shows participant device 1201 that mayjoin a session at 1203. The first participant device may be associatedwith a first participant. The first participant may, in some scenarios,be the host participant. At 1205 the system receives application inputs.Application inputs may, for example, be executed by the firstparticipant on device 1201. Application inputs may include directions toinitiate or terminate an application. Application inputs may includemodifications to a size, shape, position, and/or content of anapplication. At 1207 the server may coalesce events from a session.Events may include inputs. Events may also include updates to anapplication. The updates may sometimes occur automatically.

At 1209, the system queries whether the event should propagate tomultiple participants. This may depend on system settings. The settingsmay be preset. The settings may be default settings. The settings mayhave been set prior to, or during, the session by the host. If the eventis not to be propagated to other users, the system may propagate theevent to a persistent state store for the user at 1211. This may be usedas a setting for the current, or future sessions. This may be part ofdata storage for feedback and replay after a session.

If the event is to be propagated to other users, the system may, at1213, organize one or more events. These may include updates to size,position, content, etc. The system may propagate the events topersistent state for the room in general at 1215. The system may also,at 1217, propagate the events to a communication bus that will reachother participant devices, such as devices 1219-1223.

At 1225, the system may send update hints to the user. At 1227, thesystem may edit events edited remotely, to match the local user view.For example, if a different user with a different window size (e.g.,that user is on a tablet or a phone, and the current user is on a device1201, a laptop computer) executes a modification that is to bepropagated across all devices, the current device may transform thosemodifications to match the local view.

FIG. 13 shows illustrative system diagram 1300 in accordance withprinciples of the disclosure. System diagram 1300 shows participantdevices 1301 and 1303. The devices are shown participating in a remotecollaborative session. The screens of the devices are displaying ashared browser. The shared browser may be showing identical content onthe two devices, including image 1305 and corresponding image 1307, andtext 1309 and corresponding text 1311. However, system diagram 1300 alsoshows a portion of the shared browser which may be in a privacy orsecurity mode. Text 1313 may be associated with sensitive informationthat may be associated with device 1301. Accordingly, the platform may(automatically or in response to a request) prevent that sensitiveinformation from being shown (or, in some embodiments, from beingmodified) in corresponding text field 1315 on device 1303.

FIG. 14 shows illustrative system diagram 1400 in accordance withprinciples of the disclosure. Diagram 1400 may include central server1401 and participant devices 1403 and 1405. Diagram 1400 illustratesthat device 1405 may be connected to server 1401 with two-way connection1411. The two-way connection may allow for uplink and downlink so thatdevice 1405 can receive updates to the shared browser and transmitactions of its own back to the shared browser. Device 1403 is furthershown with uplink 1407 and downlink 1409. Uplink 1407 may transmitcredentials to the central server. Downlink 1409 may receive browsingdata for local storage at device 1403. Uplink 1407 and downlink 1409 maybe encrypted, secure, channels.

FIG. 15 shows illustrative system diagram 1500 in accordance withprinciples of the disclosure. Diagram 1500 shows some of the mechanismsfor coordinating actions performed on different devices for the sharedbrowser. Diagram 1500 shows a performance timeline including: Action Aperformed on Location A when the browser is at State 1 (1501); Action Bperformed on Location B when the browser is at State 2 (1503); andAction C performed on Location C when the browser is at State 3 (1505).As an example, Actions A and B may be typing inputs and Action C may bea mouse click. Actions A-C may be performed on different participantdevices

The platform may implement a transformation in actually applying ActionsA-C to the shared browser. Diagram 1500 shows an application timelineincluding application of Action A to Location A when the browser is atState 1 (1507). Because Action A is first, it may be appliedimmediately, and the browser may typically be in the same state atapplication as it was when the action was performed. The platform maythen apply exclusive browser focus for Action A (1509). The exclusivefocus may extend until the browser is at State 4, which may be afterState 2 (the state at which Action B is performed). As a result, ActionsB may be buffered. The platform may apply Action B at State 4 (1511).However, Action B may be applied to Location D, even though it wasperformed on Location B, because the platform may compute that LocationB at State 2 may be equivalent to Location D at State 4 (e.g., thecontent in the screen may have shifted or been scrolled). The platformmay then apply exclusive browser focus for Action B (1513). Theexclusive focus may extend until the browser is at State 5, which may beafter State 3 (the state at which Action C is performed), so Action Cmay also be buffered. When the exclusive focus is lifted at State 5,buffered Action C may be applied (1515). Action C may be applied toLocation E, even though it was performed on Location C, because theplatform may compute that Location C at State 3 may be equivalent toLocation E at State 5. Because Action C is a mouse click, exclusivebrowser focus for Action C may be shorter than that provided for typinginputs, or, in certain embodiments, nonexistent.

The steps of methods may be performed in an order other than the ordershown and/or described herein. Embodiments may omit steps shown and/ordescribed in connection with illustrative methods. Embodiments mayinclude steps that are neither shown nor described in connection withillustrative methods.

Illustrative method steps may be combined. For example, an illustrativemethod may include steps shown in connection with another illustrativemethod.

Apparatus may omit features shown and/or described in connection withillustrative apparatus. Embodiments may include features that areneither shown nor described in connection with the illustrativeapparatus. Features of illustrative apparatus may be combined. Forexample, an illustrative embodiment may include features shown inconnection with another illustrative embodiment.

The drawings show illustrative features of apparatus and methods inaccordance with the principles of the invention. The features areillustrated in the context of selected embodiments. It will beunderstood that features shown in connection with one of the embodimentsmay be practiced in accordance with the principles of the inventionalong with features shown in connection with another of the embodiments.

One of ordinary skill in the art will appreciate that the steps shownand described herein may be performed in other than the recited orderand that one or more steps illustrated may be optional. The methods ofthe above-referenced embodiments may involve the use of any suitableelements, steps, computer-executable instructions, or computer-readabledata structures. In this regard, other embodiments are disclosed hereinas well that can be partially or wholly implemented on acomputer-readable medium, for example, by storing computer-executableinstructions or modules or by utilizing computer-readable datastructures.

Thus, methods and systems for collaborative remote interactive platformsare provided. Persons skilled in the art will appreciate that thepresent invention can be practiced by other than the describedembodiments, which are presented for purposes of illustration ratherthan of limitation. The present invention is limited only by the claimsthat follow.

What is claimed is:
 1. A method for providing a remote interactiveexperience via a digital platform, sai d method executed by computerexecutable code stored in a non-transitory memory and run on aprocessor, said method comprising: receiving instructions from a hostdevice to configure a virtual room, said virtual room that is configuredto be accessible to one or more participant devices; receiving, from afirst participant device, a request to access the virtual room; and inresponse to the request, rendering a canvas on a screen of the firstparticipant device, said canvas comprising a customizable graphicalrepresentation; wherein: the canvas is configured at least in part basedon the instructions of the host device; the instructions comprise aselection of one or more interaction apps, from a set of interactionapps, for inclusion in the canvas; the one or more participant devicesare associated with a level of access, said level of access that isbased at least in part on the instructions, and said level of accesswhich comprises a set of rules that define, for each of the one or moreparticipant devices: which interaction apps are included in the canvas;which interaction apps the participant device is permissioned toactivate; and a level of permission of the participant device to modifyan interaction app; and the interaction apps include a coordinatedwindow, said coordinated window that is: activated by the host device orthe first participant device; initiated with a layout and a content,said layout comprising a size, a shape, and a screen position, and saidcontent comprising words and images that are displayed within thecoordinated window; rendered on the canvas of the host device and on thecanvas of the first participant device with the same layout and content;and when any one of the host device and the first participant deviceexecutes an action to modify the layout or the content of thecoordinated window, the modification is applied to the coordinatedwindows on the canvases of both devices.
 2. The method of claim 1further comprising: receiving requests, from a plurality of otherparticipant devices, to access the virtual room; in response to therequests, rendering a canvas on the screens of the other participantdevices, and when any one of the devices executes the action, themodification is applied to the coordinated windows on the canvases ofall the devices.
 3. The method of claim 1 wherein the content comprises:a pdf file, an image file, a web browser, or a shared document with atext editor.
 4. The method of claim 1 wherein the first participantdevice is only permissioned to activate or modify the coordinated windowwhen the host device selects an option to provide said permission. 5.The method of claim 1 wherein the canvases are configured to include atleast two interaction apps that are video tiles, and at least oneinteraction app that is a coordinated window.